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Abstract 

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network 
model is studied for arbitrary temperature. By employing a probabilistic signal-to-noise 
approach, a recursive scheme is found determining the time evolution of the distribution 
of the local fields and, hence, the evolution of the order parameters. A comparison of 
this approach is made with the generating functional method, allowing to calculate any 
physical relevant quantity as a function of time. Explicit analytic formula are given in 
both methods for the first few time steps of the dynamics. Up to the third time step 
the results are identical. Some arguments are presented why beyond the third time 
step the results differ for certain values of the model parameters. Furthermore, fixed- 
point equations are derived in the stationary limit. Numerical simulations confirm our 
theoretical findings. 

1 Introduction 

Recently, we have studied the Blume-Emery-Griffiths (BEG) neural network dynamics at 
zero temperature using the probabilistic signal-to- noise analysis (SNA) pQ. The interest of 
such a study arises from the fact that the BEG-model optimises the mutual information 
content for three-state networks In this way the best retrieval properties including, 

e.g., the largest retrieval overlap, loading capacity, basin of attraction and convergence time, 
are guaranteed in comparison with other three-state networks studied in the literature 
(31 . As has been pointed out in [I] the system allows for new stable stationary states, 
i.e. quadrupolar states, when the temperature becomes high enough. Therefore, it is 
interesting to study the dynamics of the model at arbitrary temperatures. This can be 
done in the framework of the SNA by introducing auxiliary thermal fields [Sj to express 
the stochastic dynamics within the gain function formulation of the deterministic dynamics. 
The calculations are rather involved because the feedback in these systems [@] causes the 



appearance of discrete noise, besides the Gaussian one, involving the neurons at all previous 
time steps. This prevents a closed-form solution of the dynamics although it allows for a 
recursive scheme to find the order parameters, starting from the time evolution of the 
distribution of the local fields. Hence, it seems worthwhile to also apply the generating 
functional approach (GFA) [7j, [S] to solve this feedback dynamics. This method enables 
us to find all relevant physical order parameters at any time step via the derivatives of the 
generating functional. 

Both methods are then compared for the first few time steps and the results are analyt- 
ically shown to be identical up to the third time step. Numerical simulations are performed 
for systems up to 7000 neurons and they confirm the analytical results we have obtained. 
Beyond the third time step it is found numerically that the SNA approach certainly leads 
to different results than those obtained through simulations for values of the model pa- 
rameters corresponding to spin-glass behaviour of the system. This is traced back to the 
treatment of the residual overlap representing the influence of the non-condensed patterns 
in the time evolution of the system. Finally, as in the T = case, we derive fixed-point 
equations in the stationary limit and compare them with existing results within a replica 
symmetric thermodynamic approach [3]. 

The paper is organized as follows. In Section 2 we introduce the BEG model and 
the order parameters of interest. Section 3 is devoted to the solution of the dynamics of 
the model with the SNA. The first few time steps are obtained for the main overlap, the 
neural activity, the activity overlap and the variance of the residual overlaps. Moreover, 
the stationary limit of the dynamics is discussed. Furthermore, numerical simulations of 
this dynamics are presented for the first three time steps showing agreement with the 
theoretical findings. Section 4 discusses the GFA approach to the BEG dynamics. In 
Section 5 we compare both methods analytically up to the third time step and numerically 
beyond that. Some concluding remarks are given in Section 6. All technical details are 
refered to Appendices. 



Consider a neural network consisting of N neurons which can take values {<7j|i = 1, . . . , N} 
from the discrete set S = {s^l k = 1, 2, 3} = {—1, 0, +1}. The p = aN patterns to be stored 
in this network are supposed to be a collection of independent and identically distributed 
random variables (i.i.d.r.v.), {£f |/i = 1, . . . ,p;i = 1, . . . , N} with a probability distribution 



2 The BEG model 



p(tf ) = ^ W - 1) + + 1) + (i - <W) 




with a the activity of the patterns so that 




(2) 
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Given the network configuration at time t, o"7v(t) = Wj{t)\j = 1, • • ■ ,N}, the neurons are 
updated according to the stochastic parallel spin-flip dynamics defined by the transition 
probabilities 

p , , u ,,s c c , exp{-f3ei[s k \(T N (t)]} 

Prob {ai{t + 1) = s k € S\cr N (t)) = = — p- . 3) 

The configuration <ttv(0) is chosen as input. The energy potential €i[s\cr n (t)] is defined by 

ei[s\a N (t)] = -shi(a N (t)) - s 2 9i{a N {t)) , (4) 
where the following local fields in neuron i carry all the information 

hi(a N (t)) = Jij°j(t), Oi(a N (t)) = Kijrf(t) . (5) 
The synaptic couplings Jij and Kij are of the Hebb-type 

^ = ^vE^ *« = ^X>W (6) 

/i=l ^=1 

with 

tf = |((tf) 2 -»). S = a(l-a). (7) 
The order parameters relevant for the present discussion of this model are 



where is the retrieval overlap, l^(t) related to the activity overlap, and qN(t) the neural 
activity [2], 



3 The SNA for arbitrary temperatures 

3.1 Recursive dynamical scheme 

In PP we discussed the SNA for the BEG model at temperature T = 0. In order to generalize 
this approach to finite temperatures we introduce auxiliary thermal fields jS] in order to 
express the stochastic dynamics within the gain function formulation of the deterministic 
dynamics. Consider for each i and t the auxiliary thermal fields, ^?i(t) = {ipl(t)\ I = 1,2, 3}, 
which are i.i.d.r.v. with respect to i and i, such that [5] 

Probst + 1) = s k G S\a N (t)\ = 



Y[ e(e*[s,|<M*)] - 4(t + 1) - ei[sfcl<M*)] + + !)) 



(9) 
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with [-]^ the expectation value with respect to S&i(t). The joint probability density of the 
*&i(t) with respect to I follows from comparing ©-® with Q and the usual normalisation 
to 1. The result is 

M*.M1 = 0* -gtS^d®U (±mt)) . (io) 



{£f.i«p|-»!(t)]} s 



J=l 



To make the link between the stochastic dynamics and the deterministic dynamics in a 
convenient way we observe that for each realization of the Stfi(t + 1) with density /«(•) the 
network state evolves according to the deterministic rule 

3 

(n(t + 1) = s k II ©fehkiv(i)] - 4(t + 1) - eikkTv(i)] + ^?(t + 1)) 

k=l l^k 

= g*[hi(.(rN(t)),6i(.<rN(t))]- ( n ) 

In this way the problem of deriving recursion relations for T / becomes tractable. We 
start from the following initial conditions 

E[a i (0)}=0, Var[a i {0))=q o , (12) 
E[tfa j (0)}=8 i , j 6^ l m 1 a, m\ > 0, £[<af(0)] = 8^1$ . (13) 

So we assume that the initial configuration is correlated with only one (i.e. the condensed) 
stored pattern. By the law of large numbers (LLN) Eqs.(JHJ) and (|12j ) -(|lHj ) determine the 
order parameters m^(0),qj^(0) and /^(O) at t = in the limit N —* oo. In the following 
we leave out the pattern index 1. 

For a general time step we find from Eq. ^llj) and the LLN in the limit N — > oo for the 
order parameters (eqs.fJHJ)) 

m\t + l) ^ \[((fa{ht(t)Mt))))] 9 (14) 
q(t + l) = [((AMQMt))))]* (15) 

i^t + i) v = [((mglHt)Mt))))U (i 6 ) 

where hi(t) = lim7v-*oo hi(cr^(t)) (with an analogous formula for 6i(t)), and where the 
convergence is in probability. In the above the average, denoted by ((•)), has to be taken 
over both the distribution of the {£f } and the {<7j(0)}. Note that the {cj(0)} is hidden in 
the local fields through the updating rule (fTT|) . For the Eqs. H14 |) -([16 |) the average over the 
thermal fields can be done explicitly by using the relations leading to 

m\t + l) V ^ ^((SiVpQiiWMt)))) (17) 

q(t + l) = ((W fi (hi(t)A(t)))) (18) 
l\t + l) ^ {{ m W p {hi{t),9i{t)))) (19) 
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with 



Vp(hi(t),9i(t)) 



Wp{hi{t)A{t)) 



smh((3hi(t)) 



\ exp (-P0t(t)) + cosh(f3hi(t)) 



cosh(/3hi(t)) 



(20) 
(21) 



\ exp (-06i(t)) + cosh((3hi(t)) ' 

In order to obtain the recursion relations for the local fields we can then follow the derivation 
of the zero temperature case P (see Section 3) step by step. We do not repeat this 
calculation here but write down the final results 



hit + 1) = -m(t + 1) + Xh(t)\ hi{t) - -&m(t) + -ai{t) \ + M 0, -^q(t + 1] 
0i(t + 1) = THl(t + 1) + xe(t){0i(t) - Vi l{t) + |^ 2 (*)} +^(0, £q(t + 1)) • 



(22) 
(23) 



From this it is clear that the local fields at time t + 1 consist out of a discrete part and a 
normally distributed part 



hi(t) = Mi(t) +M(0,aaD(t)) 
9i(t) = Li(t)+M{0,"E(t)) 



(24) 
(25) 



where 



r £■ a i £■ 

Xhit) Mi(t) - 2lm(t) + -<7i(t) + -m(t + 1) 
L a a " J a 



(26) 
(27) 



Mi(t+1) 

Li(t + 1) = x*(t)[.£i(t)-t^(t) + ~0?(t) + »^(t + l 
L a J 

with -D(i) and E(t) the variances of the residual overlaps defined by 

3 3 

and taking into account the noise produced by the non-condensed patterns in the local 
fields. They satisfy the recursion relations 



D(t+1) 
E(t + l) 



q(t + l) 

a 3 
q(t + l) 



+ Xe(t)E(t) + 2 X e(t)Cov[P(t), s»(t)} . 



(29) 
(30) 



In these expressions, the modified residual overlaps F*, s^, n > 1 are introduced because in 
the residual overlaps the local fields strongly depend upon ^ and rff respectively. There- 
fore, the local fields for finite N have been modified as follows 



KM = h N ,i(t) 



i 



9$rM = Nti (t) 



1 



(31) 
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such that, employing the central limit theorem (CLT) 

f(t) = lim ^^e>fe(t),^(t)]^AT(0,^±^) (32) 

S"(t) = hm ^^^|[^(t),^.(t)]^AA(0,^±^), (33) 



TV 

where we have now assumed that the r^,s^,fi > 1 depend only weakly on and rt 
respectively. We remark that in the thermodynamic limit the density distributions of the 
modified local fields hi(t) and 6i(i) equal the density distributions of the local fields hi(t) 
and 9i{t) itself. The Xh(t) an d Xe(t) are the susceptibilities corresponding to the fields 
hi(t) and 9$) 

*w - i((/tf/^ B( t^( »?i ) 
„ w = i((/^/^p tl( „w w , ( „w ( g^£W) ) 

In these expressions, Phi(t)i x ) an d Pei(t)( x ) are the probability densities of the local fields 



(34) 

(M) ' ' 



(35) 



^ )(2) = / ( n ^) Vdet ^ Ch) Vdet ( 27r 



C e) (36) 



exp ( - ixC ft V - lyC, V f ) " " 



. t-2 l 

P6i(t)(z)= / (fjdx s dy s )dx t dy t — ===== 
J „_n v det (x 



s=0 

X 



V^det (27TC,,) ^/det (2vrC 6 



exp ( - - IyC, V) *(* - U{t) - JjE(t)y t ) 



(37) 



with x = (xo,-- - ,xt~2,xt) and y = (yo,-" ,Ut-2,yt) two sets of correlated normally 
distributed variables which we choose to normalize. The correlations matrices between the 
different time steps are then given by the elements C/ l (t,t / ) = E[xtx t >] and Ce(t,t r ) = 
E[y t w]. 

This concludes the explanation of the recursive scheme from which the order parameters 
of the system can be obtained. A practical difficulty that remains is the calculation of the 
explicit correlations at different time steps. As an illustration we present the first few time 
steps in Appendix A. 
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3.2 Stationary equations 



Stationary equations for non-zero temperature can, in principle, be obtained from the 
zero temperature expressions obtained in pLI by introducing auxiliary thermal fields and 
averaging over them. However, extreme care has to be taken concerning these thermal 
fluctuations. The point is that in the case of the stationary limit at zero temperature 
it is well known that there is no difference beween the neural activity and the Edwards- 
Anderson order parameter, but for non-zero temperatures they become distinct. Therefore, 
in order to obtain the stationary limit, we assume that for t — > oo the local fields can be 
replaced by their thermal averages 

lim [(7?(* + l)]« = lim [gl (^(t) , Oft))]* « lim [<$ ([/»»(*)]*, [ft (*)]*)]* n = 1,2. (38) 

The first step is to find the thermally averaged fields. Starting from the residual overlaps 
(|2*%|) we write (we focus on the ^-quantities) 



WHi)]* = [^)]*+[xfc(tm 



where now 



tov= iim -J_^er[^(4i(t),^(t))]* ^^fo,i?i(t+i: 



with 

9i(*) = «[^(*)]|» 
The recursion relation for the field then becomes 



[hi(t + l)] 9 = -m(t + l) + Xh(t) 
a 



hi(t) - -m(t) + -ai(t) 
a a 



+ AT 0, 



(39) 



(40) 



(41) 



(42) 



where we have used the LLN for m(t) and Xhif)i V1Z - 



2 dg-q,(t) vr 1 



(43) 



Completely analogous equations can be written down for (6i(t + 1))^ and (s^it + 1))^ with 

PiW = «k 2 (i)]|» • (44) 
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Next, in order to obtain the stationary limit we assume that the thermal averages of 
the local fields and of the order parameters do not change in time. This leads to 

£ r ( \ a 7 a 

\h\-q, = -m+N 0, -^j- ^ + -rj h [a\^ = h+ -t] h [a\^ 

a V a\l-XhYJ a a 

[e]< s , = r}l+N[0, „ 2n aPl ) + %e[o% =0 + %e[(J% 
V ° (i - Xe) J a a 

with r) x = Xx/0- ~ Xx), x = h,0. The solution for [<r]$ is a self-consistent equation of the 
form 



Wh = Vp lh + -- nh [o\n l ,6 + -r l o[<r]*) ■ (46) 
V a a I 

In order to solve this equation we consider the case T = 0. There the averages over the 
thermal noise disappear, yielding 

lim V p (h, 9) = sign(/i)e(|/i| + 9) (47) 

and a Maxwell construction ^U] leads to 

a = sign (/j) 0(| /i | + 9 + A) (48) 

where 

1 / a a \ ... 
A =o \-Vh+~ve • (49) 



2 V a a 

We introduce then a set of auxiliary thermal fields and average over them to arrive at the 
fixed point equations in the limit N — > oo (dropping the index i) 



m 



with 



i J Dz J ' DyVp(h(z),9(y) + A)\\ (50) 

vJdzJ DyWp(h(z),9(y)+A)\\ (51) 

J Dz J DyWp{h{z),9{y)+A)\\ (52) 

= ^=((/ Dzz j DyVp(h(z),9(y) + A)^ (53) 

X* = ^={^j DzJ DyyW p (h(z),9(y)+A)^ (54) 



= i m + Z; = 7// + - y , (55) 

« a(l-Xft) a(l-Xe) 
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and q\ and pi given by 



Pi = 



Qi = 




I 
J 



Dz 



Dz 



I 
J 



DyV$(h(z),e(y) + A) 
DyW 2 Jh(z),9(y)+A) 




(56) 



Furthermore, in agreement with the fluctuation-dissipation theorem 



Xh = ~(q ~ qi) 



Xe = -(q-pi) 



(57) 



with Xh and xo the susceptibilities defined before, in the stationary limit. It is interesting to 
remark that the equations (|50 p -(|54 [) are the same as the fixed-point equations derived from a 
replica symmetric thermodynamic approach with a hamiltonian for sequential dynamics 4 . 

3.3 Numerical results and simulations 

The results derived in Sections 01151 and apppendix A have been studied numerically and 
compared with simulations for systems up to N = 7000 neurons averaged over 500 runs. 

As in the T = case the first few time steps, given by the explicit formula in appendix 
A, and the simulations agree very well over the whole range of a. As an illustration we 
refer to Fig. 1 presenting the order parameters as a function of a for uniform patterns, 
m o = Iq = 0.6, qo = 0.5 and T = 0.5. We remark that the critical capacity for this system 
is a c ~ 0.06 ([!])■ We then learn that the first time steps do give us a reasonable estimate 
for the critical capacity, especially through the order parameter I. 

In Fig. 2 we examine the order parameters m and I in the quadrupolar phase (m = 
0, 1 > 0) versus the paramagnetic phase (m = 0, 1 = 0), for several values of a. We see that 
a few time steps do give us already the characteristic behaviour. When time increases m 
decreases while I differentiates between the phases, as is seen in the theoretical results as 
well as in the simulations. For the quadrupolar phase {a = 0.001) I increases, deep inside 
the paramagnetic phase (a = 0.1) I decreases, while in the intermediate region (a = 0.01) 
the rate of increase of I quickly diminishes and I itself goes to zero. 

At this point, we remark that there is a small but visible discrepancy between the theory 
and simulations especially in ^(3). It is of the order O(10 -3 ) and attributed to finite-size 
effects. In this respect we report that earlier calculations ^2] of further time steps of 
the overlap order parameter in the Hopfield model also showed a discrepancy between 
simulations and numerical results, e.g., for a = 0.10 the difference m^(3) — m S i m (3) is 
again of the order O(10 -3 ). However, for 0.1 < rriQ < 0.4, the difference m^(4) — m s j m (4) 
is about 0.018 and always positive both for iV = 6000 and N = 7000. This, and the fact 
that the SNA does not give a closed form solution of the dynamics, has motivated us to 
look in Section 4 at a second, alternative approach for solving the dynamics. 
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0.64 



0.03 0.06 a 0.09 0.12 0.15 

Figure 1: Order parameters m(t), l(t) and q{t) as a function of the capacity a for the first three time 
steps for uniform patterns (a — 2/3), mo = lo = 0.6, go = 0.5 and T = 0.5. Theoretical results (solid lines) 
versus simulations (time 1, 2 and 3 represented by a circle, a plus respectively a times symbol) are shown. 

4 The generating functional approach 

The second approach we use to discuss the dynamics of the BEG-model is an extension 
of the generating functional approach (GFA) [Jj [Ej (for a recent review see, e.g., ^Hj and 
references therein). It is an exact procedure based on the idea of looking at the probability 
for finding a microscopic path in time 

t N 

Prob[rr jv(0), . . . , <r N {t)\ = Prob(<r(0)) [] \[ Prob (atf) = s k G S\a N {t' - 1)) (58) 

t'=i i=i 
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1 2,3 4 5 

Figure 2: Order parameters m(t) (bottom three lines) and l(t) (top three lines) as a function of time for 
uniform patterns (a = 2/3), mo = lo = 0.1, go = 0.5, T = 1.1 and several values of a. Theoretical results 
(open symbols) are shown versus simulations (full lines for a = 0.001, dashed lines for a = 0.01 and dotted 
lines for a = 0.1). 



where the transition probabilities are given by ©-© with the local fields /ij(<rjv(£)) and 
6i{&N{t)) replaced by 

hi(a N (t)) = AW® + Tfc,i(<) H*) = Yl K ii^) + 7«,i(f) • ( 59 ) 

Here the Jh,i(t) and 7e,i(t) are time-dependent external fields introduced in order to define 
a response function. 

The basic tool is then the generating functional 

Z N [%$] = /exp|-^^(Vi( S K( S ) + ^( S )a|( S ))|\ 

\ I 8=1 i=l J / paths 

= Yl ■■■ Prob[a N (0),...,<T N (t)] 
o-jv(o) a N (t) 

x exp J -i ^ (Ms)^(s) + Hs)ctKs)) } (60) 

[ g=l i=l J 

from which we can find all the relevant observables by calculating derivatives with respect 
to {ipi(t)} and {(f>i{t)} and taking these zero afterwards. In the sequel we omit the subscript 
indicating the path average. This path average of the spin, the correlation functions and 
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the response functions are given by 

C 4(j (t,0 = - Mm f'f^'f' , C, «(*,(') = " Mm f?"'*'*' , (61) 
' n ; ^,^Qdtpi(t)dtpj(t') , 3K ' #,*-o d(j)i{t)d(j)j{t') ' v ; 



■*,*^o0^(t)07 fcj -(tf) ' < n ' ; ^o9^(t)a 79j (f')' 

The generating function is averaged over all pattern realizations, i.e. over the disorder, 
before the path average is taken. In the limit N —* oo, this results in a theory with effective 
single spin local fields which, for the model at hand, are given by 



h(t) = ^m(t) + j h (t) + - V J2 h (t, s')a(s') + ^ z h (t) (62) 
a a 

s'=0 
t 

9(t) = r,l(t) + le (t) + *Y1 R ^ *>V) + VZze(t) (63) 



=o 



where the noises Zh(t), zg(t) are a set of correlated normally distributed variables with the 
measure 



w{z h ,z e ) 



exp {-hT,lJ=o Zhi^h 1 ]^, s')z h (s') -sE'JzzO^Wtw/Ka.^M} 



V(2tt)* det(w h ) V(27r)* det(w fl ) 

(64) 



w A = (aI-G h )- 1 C A (oI-G!:j (65) 

vro = (al- Go)' 1 C e (al-Giy 1 . (66) 
The retarded self-interactions appearing in ()62 [) -(|63 [) are then given by 

R h = (aI-G h y 1 G h , K e = (aI-G e )- 1 G e . (67) 
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Furthermore, the order parameters can be written as 



1 8Zn\^ $1 1 

m(t) = i lim lim -L V ' =- ((fr(t))) (68) 



l(t) = i lim lim =((w 2 (t))) (69) 

-)2" 



C fc (t,0 = - lim lim lrl^ = (( ff (i),(/))) (70) 

V2" 



G.(M') = Uk lim ^3®! = //!^ (72) 
7 JV-oo*,$^oiV^^(t)97 M (t') 1 7/ 



^(MO = i to lim ^T l ^f^% =((^^))) (73) 

where the overline denotes the disorder average and ((•)) denotes the averages over the noises 
Zh,zg, and the initial conditions and the condensed pattern, according to the measure 

((f[a{0),...,a{t)})) = l^f dz h dz e w[ Zh ,Z0]Proh[a(O),...,a(t)\z h ,z } f[a(0),...,a(t)] 

(74) 

Here 

Pvoh[a(0),...,a(t)\z h ,z 9 ] 

V- P w /n „ ^ exp (Ms + l)h(s) + (3a\s + l)6(s)) 
\(o£ W ^^^^n 2«p W .))ooah W .)) + l (?5) 

with /i(s) and #(s) given by (|6*2*)) and (|f)3")) . 7h,7e = 0. For more details of this calculation 
we refer to Appendix B. 

For later convenience we remark that Gh(t,t') = for t < t' due to causality (analo- 
gously for Ge(t, t')), and that for t > t' 

G h (t,t') = P((cx(t)[a(t' + 1)-Vp(h(t'),e(t'))])) (76) 
G e (t,t') = P((a 2 (t)[a 2 (t' + 1)-W^h(t%e(t'))}}} . (77) 

This set of equations completely solves the dynamics for the BEG-model. We calculate the 
first few time steps in Appendix C, in order to compare the results with those of the SNA 
method. 



13 



5 Comparison between SNA and GFA 



By comparing the explicit calculations described in the Appendices A and C for the first 
two time steps in the dynamics of the BEG model we find that they are identical and that 
in the GFA response functions, restricting ourselves for the moment to the /i-field, 

G fc (l,0) = axh(0) G fc (2, 1) = oxh(l) • (78) 

The third time step requires a bit more work. Looking at the response function in general 
(recall (|76|)) and considering t' = t — 2, the sums over a(t) and a(t — 1) can be done 
explicitly because of the specific time behaviour of the local fields, leading to 

G h {t,t-2) = 0. (79) 

Using this, together with the causality requirement on Gh and noting that a similar treat- 
ment is valid for Gg, we find that the result for the third time step in the GFA method 
coincides with the corresponding SNA result. Comparing the next time steps explicitly is 
more cumbersome. Also from the local fields as expressed by Eqs. (|22jl - l|23Jl in the SNA 
approach and Eqs. (j62j) - (|63j) in the GFA method no immediate obvious analytic connection 
is apparent. Therefore, a separate detailed analytic study of these relationships for the 
simpler Hopfield model is appropriate 11. 

In the sequel we present some strong numerical evidence that the SNA and the GFA 
approach will lead to slightly different results for certain values of the model parameters, 
specifying spin-glass behaviour, from the fourth time step onwards. This is consistent with 
our remark at the end of Section 3 that a difference, although a very small one, has been 
noticed between the SNA approach and simulations at zero temperature in the fourth time 
step of the parallel Hopfield dynamics ^2j. This result was not entirely conclusive because 
of the possible entanglement with finite-size effects. 

Reviewing the SNA approach one knows that one of the basic ingredients is the intro- 
duction of a modified residual overlap f(t) which converges to a normal distribution in the 
thermodynamic limit (recall eqs. I|32| ) -(|33| ) ). 

We have done some numerical experiments for different values of the model parameters 
comparing this limiting normal distribution in the SNA method with simulations for differ- 
ent time steps. Some results are shown in Fig. 3. We compare the normal distribution with 
simulations for time steps t = 2 and t = 9 for systems with N = 2000 neurons averaged 
over 250 runs for the initial conditions mo = Iq = 0.6, qo = 0.5, a = 2/3 and temperature 
T = 0.2 as a function of a. We note that the critical a for these parameter values is 
a c = 0.086 [1]. 

We conclude that in the retrieval region (a c < 0.086) the simulations results coincide 
quite well with the limiting distribution, while in the spin-glass region, certainly from 
a ~ 0.11 onwards, the results for t = 9 start diverting systematically. We remark that 
a set of analogous results can be obtained for s(t). This may indicate that for t > 3 
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oc=0.03 a=0.07 




Figure 3: Simulations of the distribution of the residual overlap r(t) for time steps 2 and 9, compared with 
the limiting normal distribution (full and dotted lines) as a function of the capacity a. The simulations 
are done for N = 2000 neurons averaged over 250 samples. The initial conditions for the dynamics are 
m = l = 0.6, go = 0.5, a = 2/3 and T = 0.2. 
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the SNA works quite well in a large part of the retrieval phase since most of the time 
the system reaches the attractor already after a few time steps. However, in the spin-glass 
phase the SNA method does not estimate correctly the noise of the non-condensed patterns 
represented by f(t), suggesting that the assumption on the residual noise discussed above 
is no longer fulfilled. Resolving these issues analytically is not immediately straightforward 
and beyond the scope of the present work. They will be worked out first for the Hopfield 
model mi- 

6 Concluding remarks 

The parallel dynamics of the fully connected BEG neural network model for finite temper- 
atures has been solved using both the signal to noise analysis and the generating functional 
approach. It is shown that both methods give identical results up to the first three time 
steps. 

Furthermore, by studying the distribution of the residual noise in the signal to noise 
analysis numerically and comparing it with further time steps up to t = 9 it is found that 
the signal to noise method as it is used in the literature does not estimate correctly the 
modified residual overlaps in the spin-glass region. However, in a large part of the retrieval 
region it can be considered as accurate. Numerical simulations confirm these findings. The 
signal to noise approach can be made exact by carefully studying the long time correlations 
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A The first three time steps in the SNA 

Employing the general recursive scheme of Section 3 evolution equations are derived for the 
first three time steps of the dynamics in the SNA approach. The following initial conditions 
are taken 

rn\0) = m , ^(0) = l , q(0) = q , (80) 

hi(0) = ^m +AA(o,^f), 9 i (0)=r 1 }l +^(o,^fj, (81) 
a V or / V a z / 

D{0) = %, E(0) = ^. (82) 

From now we forget about the superscript 1 to indicate the condensed pattern. Further- 
more, we do not write down all details but we refer to for the analogous T = case. 
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A.l First time step 

Prom (J17|) - (|19|) we immediately find 



m{l) = - a {l^J DzJ Dy V^h' (z),9' (y))^ , (83) 
?(1) = ((J Dz J 'DyWp{h' (z),e' (y)))), (84) 
1(1) = (LjDzjDyWp{h' Q (z)X{y))\), (85) 



h' (z) = ^mo + y/aaD(0)z, 9' (y) = r,l + J ^E(0) y (86) 



where 



and where Dx = dx exp(— x 2 /2) / ^/~2~k is the Gaussian measure. Because of the modification 
of the local fields we know that 

Cov[^(0),r^(0)] =£[^(0)^(0)] = S[i^(O)^(^(O),0f(O))] (87) 

Cov[^(0),^(0)] =^[^(0)^(0)] = E[^af(0)gl(hf(0),ef(0))] (88) 

Using the recursion relations 1|29|) - 1|3U|) we obtain 

D(l) = ^ + x l(0)D(0) + 2 Xh (0)R(l,0) (89) 

E(l) = ^l + x j(0)E(0)+2 Xe (0)S(l,0), (90) 
a 

where the correlation parameters R(t,t') and S(t,t') are defined by 

R(t,t') = ±E[gy(h(t-l),0(t-l))gy(h(l/-l),6(t'-l))] (91) 

R(t,0) = ^E[gy(h(t-l),0(t -1))<t(0)] (92) 
or 

S(t,t') = lE[gl(h(t-l),e(t-l))gl(h(t'-l),e(t'-l))] (93) 

S(t,0) = ±E[&(Ht-l)j(t-l))o*(p)] (94) 

Since the density distribution of h and h are the same in the limit N — > oo we have for the 
first time step 

R(1,0) = ± //a(0) JdzJ DyVp{h' Q (z)X{v))j) (95) 
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5(1,0) = l^ 2 (0) J DzJ DyWp{h' (z),e' (y))\\ (96) 



Furthermore, since there are no feedback correlations at time zero 

i/// n J n . f dVp{h{t\e{t)Y 



Xh(0) = UlJ DzJ Dy 



dh(t) 



rm 1 I' [ n f dWp(h(t),6(t)) ^ 

xe{0) = ~a\\J Dz J D n — m — ) 



(h' (z),e' (z))/ 
(h' (z),e' (z)), 



(97) 
(98) 



A. 2 Second time step 

First, we need the distribution of the local fields at time step 1 using (|24j) - ()27|) 

c ' (y. 

hi(l) = ^m(l) + - X h(0)ai(0)+M(0,aaD(l)) (99) 
a a 

9,(1) = ^(1) + ^(0K 2 ( ) +^(0,^(1)) (100) 
a a 

These results enable us to write down the important order parameters at time 2: 

m(2) = UU j Dz j ' DyV^h' x {z)My))S) (101) 

9(2) = (IJ Dz J DyWpW(z),%(y))\\ (102) 

1(2) = (L J DzJ DyW^^My))^) (103) 

defining 



h[(z) = -£m(l) + - Xh (0)a(0) + y/aaD(l) z (104) 
a a 

9[(y) = v l(l) + ^ X9 (0)a 2 (0) + J"E(l)y (105) 
a \ a 

The calculation of the variance of the residual overlaps proceeds as follows. From the 
recursion relations (|29j) - (|30j) we get 



E(2) = ^ + x l(l)E(l) + 2 Xe (l)(S(2,l)+ Xe (0)S(2,0)) (107) 



D(2) = ^ + X 2 h (l)D(l) + 2 Xh (l)(R(2,l)+ Xh (0)R(2,0)) (106) 
?(2) 
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where 



12(2,0) = ^((a(0)jDzjDyVp{h[(z),e[(y))\\ (108) 

5(2,0) = \^ 2 (0) J Dz J ' DyW p {h{{z),e' 1 {y))fj (109) 

To obtain the further correlations introduced we remark that because a (I) appears in 
the expressions, we have to introduce the correlation between time steps and 1. The 
relevant correlation coefficients are given by 

K ' ^aaD{t)^/aaD{t>) V ' 

= Ejm - mum - u m 

and the joint distribution 

Duj a,b (z x dzdy ( z 2 -2zyp x (a,b) + y 2 \ 

Then we arrive at 

#(2,1) = 1 //y Du 1 h ' (z,s)D^°( y ,t)V l3 {h' 1 (z),9' 1 ( y )) V^h' (s)X(t))\) (113) 

5(2,1) = l(Q ^r(^^r(^O^WW^Uy))^W(s)^o(0))) (H4) 

Finally, the susceptibilities at time step 1 are given by 



dh(t) 



(115) 
(116) 



A. 3 Third time step 

We have all the quantities needed to write down the local fields at time t = 2 

hi(2) = ^m(2) + - X ft(l)(<7i(l) + Xft(0M0))+jV(0,aa£)(2)) (117) 
0,(2) = r ? ^(2) + ^(l)(^(l)+ X(? (0)a 2 (0))+AA(0,|i?(2)) (118) 
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This gives for the order parameters 



77? 



(3) = D^°(z,s)Duj 2 e ' (y,t) £ Vp[ti 2 (z),9' 2 (y)) Up(ti (s),9' (t),a(l))\\ 

(119) 

g(3) = (^J Du 2 h '°(z,s)Du;f (y,t) £ Wp{h 2 (z),9 2 (y)) UM( S )X(t),a(l))^ (120) 



1(3) = (^r, J Du 2 h '°(z,s)Du;f (y,t) £ W p {h' 2 {z), 9> 2 {y)) Up(h' (s),9' (t),a(l)) 



(121) 



with 



and 



T7 (h'(* B'(t) , n » gPNMjj!M 
E^(^( a ),e (t) >O -(l)) - 2exp( ^ (t))cosh(/3/l , (s)) + 1 ( 122 ) 



£ /r,\ a 



h' 2 {z) = im(2) + + Xfc(0)ff(0)) + V«a£>(2) z (123) 

^(y) = t?Z(2) + ^(l)(<x 2 (l) + X e(0)a 2 (0)) + J *E{2) y (124) 
a \ a 

In an analogous way further time steps could be calculated. 

B The complete GFA calculation 

Starting from the path (|58|) with the transition probabilities given by © the generating 
function (jBO)) can be written as 

jv t 

z[% $]=j2 - e pr ° b ( CT (o)) n n ex p h^w (^( s ) + - !)))] 

<T(0) <T(f) i=ls=l 

x exp [-iaf{s) {<pi{s) + ip9i{cr{s - 1)))] 

x exp [-In (2 exp (/36>i(o-(s - l)))cosh(/3/?, i (o-(s -!))) + !)] (125) 
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where we have left out the subscript N. We then isolate the local fields by inserting the 
appropriate delta distributions 



Z[* J $] = ^...^Prob( t r(0)) J 



dhdhdOde 



(T(0) <7(t) 



n 



x exp 



exp 



( 27r )2JV(t+l) 

TV 

Y^hi(s){hi(s) - 7w (s)) +iJ2H s M( s ) ~ 7e,i(s)) 
i=i i=i 

TV 

*i(3)(il>i(8) +iph i (s-i))-iJ2o*(s)(<lH(a)+ipO i (B-l)) 



N 



i=l 



i=l 



x exp [- ln{2exp (pe^s - 1)) cosh(/?/i;(s - 1)) + 1}] 



x exp 



TV 



TV 



i H0)hi(0) 0i(O)0i(O) 
1=1 1=1 
x exp [J 7 ] 

where exp [J 7 ] includes all J and if dependent terms 



(126) 



exp [J 7 ] 



exp 



x exp 



x exp 



E J > 



t / TV 
s=0 \ i=l 

-^e(^eW) 

s =o \ i=i / y=i 

t TV 

E E (hi{s)JiiO-i(s) + Oi(s)K u of(s 



s=0 i=l 



(127) 



We remark that in the expression above, the diagonal terms are excluded by the introduc- 
tion of the last factor, which we denote by M. 

We first concentrate on the terms containing the disorder, i.e., the terms containing 
Jij and Kij, of which we need to know the limit N —* do. Using the learning rule © and 
taking for simplicity one condensed pattern again, say \i = 1, we can study the condensed 
and the non-condensed part of the noise (|127|) by splitting 

exp [T] = exp \T C + T nc ] . (128) 

With respect to the condensed pattern it is seen that the following single time variables 
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are relevant 

N „ N 



""•• s) = -\Y,£ a ^ w = -4 ( 129 ) 



AT „ AT 



i=l i=l 



To achieve site factorisation we then introduce appropriate delta distributions for l|129|) - 
(|130|) and their conjugated variables denoted with a hat. Consequently, 

r N^t+i) 

exp [JT C ] = / dmdmdkf l dk.f l dldl(Ik.gdk., 



(2vr) 4 (*+ 1 ) 
x exp {— iN (kh • m + kg ■ 1))} 

x exp | — iN • m + k h • k h + 1 • 1 + kg ■ kg^j j 

x exp {-*EE (™(*)f W) + kis^Hs) + l(s) V }aUs) + M^ito) j (131) 

With respect to the non-condensed patterns, we first introduce gaussian integration 
Dx = r[s=o {(^ 7T ) 1 ^ 2 dx(s)exp[—^x 2 (s)]) to arrive at (we forget about superindex 1 for 
the condensed pattern) 

exp[T nc ] = (^j Dx 1 Dx 2 Dy 1 Dy 2 

x exp < -== E ~ E ( x i(^( s ) + a i( s )) + iyi{s)(hi(s) - <7<(s))) > 



i=l s=0 



r AT t 

X 6XP I 7fw ? ^ S (^fe + °Hs)) + *l/2 - a 4 2 (s)) 

L » * i=l s=0 

x exp i ia £ ^ f~^( S ')^(s) + i^(fl')of («)) <W 1 (132) 

with clear notation. We have also rewritten M in a more suitable way. Performing the 
average over the patterns and expanding the exponentials up to order 0(N~ 1 ) it is seen 
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that the following two time variables turn up 





s') - 


1 N 

i=i 


Qh(s, s') = 


1 N 

1=1 


(133) 


p(s, 


s') - 


1=1 


Qe(s,s') = 


1 * 

i=l 


(134) 


K h (s, 


s') - 


1 w 

— ^^(5)^(5') 

1=1 


Kg(s,s) = 


1 * 

i=i 


(135) 



Introducing again appropriate delta distributions we obtain after some matrix algebra 

/7V\ 12(t+1) f ~ ~ 

exp [.F nc ] = f — J / dqdqdQ h dQ h dK h dK h dpdpdQgdQgdKgdKg 

x exp [iiVTr (qqt + Q h Q h t + K h K h t + ppt + Q e Q^ + K e K e ^ 

x exp [iaNTi (K h + Kg)] 

_. t N 

~^/ZYl s')(Ti(s)(Ti(s') + Q h (s, s%{s)hi{s') + K h (s, s')ai(s)hi(s')j 

s,s' 1=1 
. t AT 

s,s' 1=1 

-| [in (Q h q - (il - Kjj(a - K h )) + In (Q e p - (il - Kj)(il - K„)) 



x exp 



x exp 



x exp 

L Z L \ /\ / J J 

(136) 

with I the unit matrix of dimension (t + 1)- 

This concludes the discussion of the averaging over the disorder with as result a gener- 
ating functional of the form Z ~ exp N... which can be evaluated in the limit N — > 00 via 
the saddle-point method. 



with 



Z[^,$]oc y"{...}exp [iV (a In f2 + H + A)] 
« = ([QhQ " (^1 " Kjj(il - K h )][Q e p - (il - Kj)(il - K fl )])" 



(137) 
(138) 



• m + k h • k h + k h • m + 1 • 1 + kg ■ kg + kg ■ ij 
+ iTr (qqt + Q h Q h t + K h K h t + ppt + Q e Q fl t + ^ gKg \ + a(Kh + Kg)) (139) 
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JV 

A = — In < / dhdhdOdO Y Prob(<r(0)) 
i=i ' <r(o)...<r(t) 



x exp 



x cxp 



x cxp 



x cxp 



x exp 



s=0 
t 



s=0 



Y 9i(s)(0i(s) - le ,i( s ) - ke(s)Vi) ~ Y In (2 exp (/?0( s - 1)) coshes - 1)) + 1 

5 = 1 

Y f <7(a)(^(s) + m(s)^ + i/?fc(s - 1)) + <7 2 (s)(&(s) + f(s)fft + i0O(s - 1)) 
=0 v a 
t 

Y q(s, s')a(s)a(s') + K h (s, s')a(s)h{s') + Q h (s, s')h{s)h{s' 



s=0 



s,s'=0 
t 



CL 

s,s'=0 



(140) 



At this point, some remarks are in order. First, we have been able to factorize the 
generating functional with respect to the sites. Further, the external fields are only present 
in A. Therefore it is useful to introduce the effective single spin measure 



</>*,* = 



j dhdhdedeY Jam . Mt) ?™h(a(Q))M l <x,&,h,h,0,0 / 



/ dhdhdodd Ea(o)...a(t) ProbHo))M a, h, h, e, e 



(141) 
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where the auxiliary fields ipi and fa have been taken to be zero and Mi is given by 

t j. N 



exp 



x exp 



x exp 



x exp 



x exp 



x exp 



£ ks){h(s) - 7h(a) - m(s)^) + i £ 9(s)(9(s) - le { 8 ) - l(s) m ) 



s=0 
t 



i=l 



In (2 exp (J39(s - 1)) cosh(/?/i(s - 1)) + 1) 



s=l 

t 



-i <7(a) i(3h(s - 1) + + a 2 (s) {i(39(s - 1) + 



^(0)^(0)^-^(0)^(0)^ 



— V a >( S )<7( S ') + Qh(s, s')h(s)h(s') + K h (s, s')a(s)h(s'] 

s,s' 

. t 

~\ E (/( s > *V(* W) + Qe(s, s>)9(s)9(s') + K e {s, s')a\s)9{s')) 



s,s 



(142) 



The parameters m, k/j, 1, kg, q, Q^, K^, p, Q#, Kg in M J...] are defined as the solutions of 
the saddle-point equation d(E + A) = 0. Furthermore, we have dropped the subindices i 
from the external fields since they are taken to be site-independent. 

Before determining the saddle-point solutions we write down the following identities 



/ dm...dK e e N{E+A)+aN ln ^+°( ln W) 



dipi{s) 



lim — - — - = lim 

*,*^o dipi(s) *,$^o f d m ...dKee N( .~+ A )+ aNln ( n )+°( ln ( N )) 



-i(v{s))*,i 



hm = -z(ct z (s)), 



(143) 
(144) 



#~¥^o dipi(s)dipj(s') 



lim 



- = - < 5y[( (7 (s)(7(s / ))*,i - (^OOM^OM - ( 14 5) 



lim 

^,<i>^o dfa(s)dfa(s') 



5 ij [{a\s)a\s')) %i - {a\s)Ui{a\s')) %i \ - (o*{8))., i (o J W)*j 

(146) 
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v,$^>o 0<pi(s)d'Yo,j{s') 



(149) 



lim ffl^ = -i(h( 8 ))»,< (150) 



lim 8 ^f} =-i(Hs))*,i (151) 
*,*-o 87^ (s) 

where we recall that the overline denotes the disorder average and where we used the 
normalization Z[0, 0] = 1, we find that the last two equations are equal to 0, since the limit 

$ — » can be interchanged with the derivative, due to the fact that the local fields are 
completely unrelated with the external fields. This leads to (h(s))*^ = 0, (^(s))*^ = 0, 
(h(s)h{s'))^i = and (0(s)0(s'))*,i = 0. 

Recalling (|61|) we then have obtained the following identities 



(o-i(a)) = <<r(s))„,i (152) 

Rg) = (° 2 (s))* 4 (153) 

Ch,ij{B,ff) = 5 lj {a{s)a{s'))^ i + {l-5 ij ){a{s))^ i {a{s'))^ j (154) 

Ce,ij(5,s') = ^■(a 2 ( S )a 2 ( S ')}* ) i + (l-(5i i )(^ 2 (s))*,i(^ 2 (s / ))*,j (155) 

G h)ij (s,s') = 5ij{a(s)h(s'))^i (156) 

G e>ij {s,s') = 5 ij {a 2 (s)9(s')), il (157) 
from which we obtain by using the LLN 

i N i N 

C h (s,s') =Km-52c h ,ii(s,s') =Vm £ £>(*M«')>.,< = «<r( S )a(/)),) $ (158) 



N ^ ' V ' ' N->oo N 

i=l i=l 

N , N 



Cg(s,s') = i fa^ C M(«.'') = M i^^W^M).,* = «^(^V)).) e (159) 



N ^ *' y ' ' N-,oo N 

i=l i=l 

JV 



G h ( S , S ') = lim i^ Gfclii ( S , S ') = lim 1 ^ 9 H.))^ = -i(< ff (a)&( fl ')>.) (160) 

2=1 ' v ' 



i=l 



ft(«y)= Km i^G Mi ( S)S ')= lim i ^ ^-^r^fs)).,. = -i ({cr 2 (s)<9(s')),\ (161) 

JV^oo JV N^oo N ' C*70 its') \ / £ 

i=l z — 1 ' v ' 
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where (•)£ denotes the average over the condensed pattern. Finally, we write down the 
saddle-point equations. Variation with respect to m, k^,l and kg gives 



m(s) = k h (s) k h (s) = m(s) l(s) = kg(s) k e (s) = l(s) 
Next, we calculate the variation with respect to m, k^,! and kg, leading to 



m(s) = <£(a( S )}*) 5 k h (s) = (f(/i(s)>«.) = 
l(s) = <r?(a 2 ( S )),) e ke(s) = (r,(§(s))^ = 

Variation with respect to q, and gives 

q(s,s > ) = -C h (s,s') Q h (s,s')=0 K h (s, a') = -G h (s, s') 
a a 

and with respect to p, Qg and Kg 

p( s ,s') = lc e (s,s') Q e (s,s') = K e {s,s') = % -G e (s,s') 



(162) 

(163) 
(164) 

(165) 
(166) 



Finally, variation with respect to of q, Qh, K^, p, Qg and K.g i leads to, after some 
algebra 



q(s,s 
Qh{s,s' 

K h (s,s' 

Qe{s,s' 



p(s,s')=0 
iaa 
2~ 



al - Gh)- 1 C h (al- G f 



era 



(al-Gh)' 1 ] (s,s')-al(s,s f ) 



iaa 
~~2 



aa 



al - Gey 1 C e (51 - Gj 
(al — Gg)" 1 (s, s') — al(s, s') 



(s,s') 



(167) 
(168) 

(169) 

(170) 

(171) 



After the introduction of the saddle-point solutions, the single spin measure ()141|) is 
expressed in terms of the physical order parameters, {m , 1, C/,, Cg, Gh, Gg}. Rewriting 
the average in the following way 

((f[a(0),...,a{t)})) =(^j dh(0)...dh(t - l)d0(O)...dO(t - 1) 

£ Prob [h(0), h(t - 1), 0(0), 9(t - 1); a(0), a(t)\ f [<r(0), a(t)} \ 

<r(0)...ff(t) / , 
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with 



Prob[...]~Prob(<7(0)) ! ]\dz h (s)dz e (s) 

t 

h(s) - 7 h (s) - m(s)£ - a ^ [(al - G ft ) _1 (s, s'M 5 ') + ~ a ( s "> ~ Vaz h (s) 



x 5 



s'=0 
t 



9(s) - y e (s) - l(s)r, -aJ2 [(^ " G 6 



a 



s'=0 



(s,s')a (s) + -zra (s) - y/az e (s) 



exp (l3a(s + l)h(s) + (3a 2 (s + l)0(s)) 



2exp ((39(s))cosh((3h(s)) + 1 



/ 



x / dhdOexp 



1 r -i 

^ (al-G^C^al-G^) 



s,s'=0 L 



(s,s')/j(s)/i(s') 



x exp 



x exp 



a 
2 



1 r -i 

2 (al - Ge)- 1 C, (al - Gj) 



s,s'=0 
t 



( S , S ')<WV) 



s=0 



(172) 



the integration over h and 6 is easy since the exponents multiplying the terms h(s)h(s') and 
9(s)9(s') are symmetric matrices. Doing the integration and including the normalization 
factors, we find the results of Section 4. 



C First two time steps in the GFA 

We use the GFA results to calculate the first two time steps of the dynamics explicitly. The 
initial conditions are chosen as follows: 7(0) = /3(0) = 0, i.e., no external field, m(0) = mo, 
1(0) = l , C h (0,0)=Ce(0,0)=q , G h (0, 0) = G e (0, 0) = G h (0, 1) = G e (0, 1) = 0. 



C.l First time step 

For time 1, the relevant probabilities are 



w[z h (0),z e (0)} 



aa 



2irq 



exp 



-Mo)) 2 



a 2 



Prob[a(0),<7(l)|z h (0),^(0)] = Prob(a(0)) 



2q 2q 
exp(/3a(l)fe(0)+/?(j 2 (l)g(0)) 
2 exp (f36(0)) cosh(/?/t(0)) + 1 



(173) 
(174) 
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with h(0) = £m /a + y/az h (0) and 0(0) = l f] + y/azg(0). Then, using (J71J) and l(75|). 



m(l) = 1 



+ 



Dz fO^Dz f(n / exp(/?0+(O))sinh(/^+(O)) 

^(0)^(0) 1 2exp (/36)+(0)) cosh{ph+m + 1 

exp (/?0+(O))sinh(/?/r(O)) 



2exp(/3#+(0))cosh(^/i-(0)) + 1 



(175) 



with, as usual, Dz a gaussian measure defined in (|173j) and where h^(0) = ±mo/a + 
yfazh(0) and + (O) = Iq/cl + T/azg(Q). In the same way we find the values for 1(1) and 
q(l), i.e. 1(1) = «w 2 (l)» and g(l) = «a 2 (l)». 
The correlation and response functions are 



(4(1,0) = c h (o,i) = <(<7(oMi)>> 

0,(1,0) = C e (0,l) = ((a(0)a(l))) 

G h (l,0) = (3((a(l)[a(l)-Vf,(h(0),9(0))})} 

G e (t,t') = (3({a 2 (l)[a 2 (l)-W(i(h(0),9(0))])) 



(176) 
(177) 
(178) 
(179) 



C.2 Second time step 

In this time step correlations start to appear. 



qo C M (1,0) 
C M (1,0) q(l) 



(al 



r<t=2 



r 



i 



(al-G 



t=2\ 



a 
G h (l,0) a 

The explicit single-site average for time 2 is determined by 

1 



a 2 



a 
G h (l,0) a 



w[z h (O),z h (l),z (O)z e (l)] 



^^(q A h -Bl)(q Q A e -B s 
1 



x exp 
x exp 



2C h 
1 

2Ce 



Prob[c7(0), a(l)\z h (0), z e (0)] = Prob(cr(0)) 



z h (0)) 2 A h + z h (l)) 2 Qo ~ 2z h (0)z h (l)B h 

(z e (0)) 2 A e + z e (l)) 2 q - 2z e (0)zg(l)B e 
exp (f3a(l)h(0) + (3 a 2 (I) 9(0)) 



(180) 



2 exp (/39(0)) cosh(/?fc(0)) + 1 
exp(/?a(2)fr(l)+/fo 2 (2)0(l)) 
2 exp ((39(1)) cosh(/3fr(l)) + 1 



(181) 
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with the local fields at time t = 1 given by 

h(l) = im(l) + -G A (1,0)^(0)+^(1) (182) 
a a z 

9(1) = l(l) v + ^G h (l,0)a 2 (0) + ^z e (l) (183) 
and the coeficients defined by 

A h = a 2 q(l)+2aC h (l,0)G h (l,0)+q (G h (l,0)) 2 (184) 

B h = a 2 C h {l,0)+aq G h (l,0) (185) 

C h = q q(l)-C 2 h (l,0) (186) 

A e = a 2 q(l) + 2aC e (l,0)G e (1,0) + q (G e (l, 0)) 2 (187) 

B h = a 2 C e (l,0) + aq o G e (l,0) (188) 

C h = ?o9(l) -C$(1,0) (189) 

From these relations any quantity can be computed. We present the final expressions 
for m(2), 1(2) and g(2). 

m(2) = \ Ua(2))) (190) 

1 /(zfc(l)) 2 , (z e (l)) 2 ' 



2n^A h ~X ( 



J dz h (l)dz g (l)exp 



+ 



2\ A h A e 
x {AV£(/i++(l),0++(l)) - CVp(h- + (l),0 ++ {l)) +DVp(h+ (l),d+°(l)) 

-Dv p (h-°(i),e +0 (i)) + cv p (h + -(i),e ++ (i)) - Av p (h-(i),e ++ (i))} (wi) 

where the coeficients A, C and D coming from the distribution 

Prob(<7(0),0 = A5(a(0) _ i) S £ _ i) + B5(a(0) - 1)5(0 

+ CS(a(0) - 1)6(Z + 1) + D6(<t(0))8(Z - 1) + ES(a(0))S(0 
+ D<5(<j(0))<5(£ + 1) + C<5((7(0) + 1)6(£ - 1) 

+ B5(a(0) + 1)5(0 + Af(<r(0) + 1)<S(£ + 1) (192) 



defined by 



A = 


j(m + (1 - a)Z + qo) 


5 = 


2 (go aZ ) 


c = 


|(-m + (1 - a)Z + go) 


£> = 


|(l-(l-a)Z + go) 


E = 


(1 - a)(l + al ~ qo) 



(193) 

(194) 

(195) 

(196) 
(197) 
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The local field contributions appearing in these final equations have the following form, 
due to the average over the initial conditions 

h uv {1) = u H^l +v ^ Gh{h0) + ^ Zh{1) (198) 
t — n cs> 

9^(1) = -—l(l) +y -G e (l,0) + V^ze(l) (199) 

where u,v £ { — 1,0, 1} and x, y £ {0, 1}. 
In the same way we find q(2) and £(2). 
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